

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
  <meta charset="utf-8" />
  
  <meta name="viewport" content="width=device-width, initial-scale=1.0" />
  
  <title>Service Management &mdash; Ceph Documentation</title>
  

  
  <link rel="stylesheet" href="../../_static/ceph.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/pygments.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/graphviz.css" type="text/css" />
  <link rel="stylesheet" href="../../_static/css/custom.css" type="text/css" />

  
  
    <link rel="shortcut icon" href="../../_static/favicon.ico"/>
  

  
  

  

  
  <!--[if lt IE 9]>
    <script src="../../_static/js/html5shiv.min.js"></script>
  <![endif]-->
  
    
      <script type="text/javascript" id="documentation_options" data-url_root="../../" src="../../_static/documentation_options.js"></script>
        <script src="../../_static/jquery.js"></script>
        <script src="../../_static/underscore.js"></script>
        <script src="../../_static/doctools.js"></script>
    
    <script type="text/javascript" src="../../_static/js/theme.js"></script>

    
    <link rel="index" title="Index" href="../../genindex/" />
    <link rel="search" title="Search" href="../../search/" />
    <link rel="next" title="MON Service" href="mon/" />
    <link rel="prev" title="Host Management" href="../host-management/" /> 
</head>

<body class="wy-body-for-nav">

   
  <header class="top-bar">
    

















<div role="navigation" aria-label="breadcrumbs navigation">

  <ul class="wy-breadcrumbs">
    
      <li><a href="../../" class="icon icon-home"></a> &raquo;</li>
        
          <li><a href="../">Cephadm</a> &raquo;</li>
        
      <li>Service Management</li>
    
    
      <li class="wy-breadcrumbs-aside">
        
          
            <a href="../../_sources/cephadm/services/index.rst.txt" rel="nofollow"> View page source</a>
          
        
      </li>
    
  </ul>

  
  <hr/>
</div>
  </header>
  <div class="wy-grid-for-nav">
    
    <nav data-toggle="wy-nav-shift" class="wy-nav-side">
      <div class="wy-side-scroll">
        <div class="wy-side-nav-search"  style="background: #eee" >
          

          
            <a href="../../">
          

          
            
            <img src="../../_static/logo.png" class="logo" alt="Logo"/>
          
          </a>

          

          
<div role="search">
  <form id="rtd-search-form" class="wy-form" action="../../search/" method="get">
    <input type="text" name="q" placeholder="Search docs" />
    <input type="hidden" name="check_keywords" value="yes" />
    <input type="hidden" name="area" value="default" />
  </form>
</div>

          
        </div>

        
        <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="main navigation">
          
            
            
              
            
            
              <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="../../start/intro/">Ceph 简介</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../install/">安装 Ceph</a></li>
<li class="toctree-l1 current"><a class="reference internal" href="../">Cephadm</a><ul class="current">
<li class="toctree-l2"><a class="reference internal" href="../compatibility/">Compatibility and Stability</a></li>
<li class="toctree-l2"><a class="reference internal" href="../install/">部署个全新的 Ceph 集群</a></li>
<li class="toctree-l2"><a class="reference internal" href="../adoption/">现有集群切换到 cephadm</a></li>
<li class="toctree-l2"><a class="reference internal" href="../host-management/">Host Management</a></li>
<li class="toctree-l2 current"><a class="current reference internal" href="#">Service Management</a><ul>
<li class="toctree-l3"><a class="reference internal" href="mon/">MON Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="mgr/">MGR Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="osd/">OSD Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="rgw/">RGW Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="mds/">MDS Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="nfs/">NFS Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="iscsi/">iSCSI Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="custom-container/">Custom Container Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="monitoring/">Monitoring Services</a></li>
<li class="toctree-l3"><a class="reference internal" href="snmp-gateway/">SNMP Gateway Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="#service-status">Service Status</a></li>
<li class="toctree-l3"><a class="reference internal" href="#daemon-status">Daemon Status</a></li>
<li class="toctree-l3"><a class="reference internal" href="#service-specification">Service Specification</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#retrieving-the-running-service-specification">Retrieving the running Service Specification</a></li>
<li class="toctree-l4"><a class="reference internal" href="#updating-service-specifications">Updating Service Specifications</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#daemon-placement">Daemon Placement</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#explicit-placements">Explicit placements</a></li>
<li class="toctree-l4"><a class="reference internal" href="#placement-by-labels">Placement by labels</a></li>
<li class="toctree-l4"><a class="reference internal" href="#placement-by-pattern-matching">Placement by pattern matching</a></li>
<li class="toctree-l4"><a class="reference internal" href="#changing-the-number-of-daemons">Changing the number of daemons</a></li>
<li class="toctree-l4"><a class="reference internal" href="#algorithm-description">Algorithm description</a></li>
</ul>
</li>
<li class="toctree-l3"><a class="reference internal" href="#removing-a-service">Removing a Service</a></li>
<li class="toctree-l3"><a class="reference internal" href="#disabling-automatic-deployment-of-daemons">Disabling automatic deployment of daemons</a><ul>
<li class="toctree-l4"><a class="reference internal" href="#disabling-automatic-management-of-daemons">Disabling automatic management of daemons</a></li>
<li class="toctree-l4"><a class="reference internal" href="#deploying-a-daemon-on-a-host-manually">Deploying a daemon on a host manually</a></li>
<li class="toctree-l4"><a class="reference internal" href="#removing-a-daemon-from-a-host-manually">Removing a daemon from a host manually</a></li>
<li class="toctree-l4"><a class="reference internal" href="#see-also">See also</a></li>
</ul>
</li>
</ul>
</li>
<li class="toctree-l2"><a class="reference internal" href="../upgrade/">Ceph 的升级</a></li>
<li class="toctree-l2"><a class="reference internal" href="../operations/">Cephadm operations</a></li>
<li class="toctree-l2"><a class="reference internal" href="../client-setup/">Client Setup</a></li>
<li class="toctree-l2"><a class="reference internal" href="../troubleshooting/">Troubleshooting</a></li>
<li class="toctree-l2"><a class="reference internal" href="../../dev/cephadm/">Cephadm Feature Planning</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="../../rados/">Ceph 存储集群</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../cephfs/">Ceph 文件系统</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../rbd/">Ceph 块设备</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../radosgw/">Ceph 对象网关</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/">Ceph 管理器守护进程</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../mgr/dashboard/">Ceph 仪表盘</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../api/">API 文档</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../architecture/">体系结构</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/developer_guide/">开发者指南</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../dev/internals/">Ceph 内幕</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../governance/">项目管理</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../foundation/">Ceph 基金会</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../ceph-volume/">ceph-volume</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/general/">Ceph 版本（总目录）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../releases/">Ceph 版本（索引）</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../security/">Security</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../glossary/">Ceph 术语</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../jaegertracing/">Tracing</a></li>
<li class="toctree-l1"><a class="reference internal" href="../../translation_cn/">中文版翻译资源</a></li>
</ul>

            
          
        </div>
        
      </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">

      
      <nav class="wy-nav-top" aria-label="top navigation">
        
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="../../">Ceph</a>
        
      </nav>


      <div class="wy-nav-content">
        
        <div class="rst-content">
        
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
           <div itemprop="articleBody">
            
<div id="dev-warning" class="admonition note">
  <p class="first admonition-title">Notice</p>
  <p class="last">This document is for a development version of Ceph.</p>
</div>
  <div id="docubetter" align="right" style="padding: 5px; font-weight: bold;">
    <a href="https://pad.ceph.com/p/Report_Documentation_Bugs">Report a Documentation Bug</a>
  </div>

  
  <div class="section" id="service-management">
<h1>Service Management<a class="headerlink" href="#service-management" title="Permalink to this headline">¶</a></h1>
<p>A service is a group of daemons configured together. See these chapters
for details on individual services:</p>
<div class="toctree-wrapper compound">
<ul>
<li class="toctree-l1"><a class="reference internal" href="mon/">MON Service</a></li>
<li class="toctree-l1"><a class="reference internal" href="mgr/">MGR Service</a></li>
<li class="toctree-l1"><a class="reference internal" href="osd/">OSD Service</a></li>
<li class="toctree-l1"><a class="reference internal" href="rgw/">RGW Service</a></li>
<li class="toctree-l1"><a class="reference internal" href="mds/">MDS Service</a></li>
<li class="toctree-l1"><a class="reference internal" href="nfs/">NFS Service</a></li>
<li class="toctree-l1"><a class="reference internal" href="iscsi/">iSCSI Service</a></li>
<li class="toctree-l1"><a class="reference internal" href="custom-container/">Custom Container Service</a></li>
<li class="toctree-l1"><a class="reference internal" href="monitoring/">Monitoring Services</a></li>
<li class="toctree-l1"><a class="reference internal" href="snmp-gateway/">SNMP Gateway Service</a></li>
</ul>
</div>
<div class="section" id="service-status">
<h2>Service Status<a class="headerlink" href="#service-status" title="Permalink to this headline">¶</a></h2>
<p>To see the status of one
of the services running in the Ceph cluster, do the following:</p>
<ol class="arabic simple">
<li><p>Use the command line to print a list of services.</p></li>
<li><p>Locate the service whose status you want to check.</p></li>
<li><p>Print the status of the service.</p></li>
</ol>
<p>The following command prints a list of services known to the orchestrator. To
limit the output to services only on a specified host, use the optional
<code class="docutils literal notranslate"><span class="pre">--host</span></code> parameter. To limit the output to services of only a particular
type, use the optional <code class="docutils literal notranslate"><span class="pre">--type</span></code> parameter (mon, osd, mgr, mds, rgw):</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><style type="text/css">
span.prompt1:before {
  content: "# ";
}
</style><span class="prompt1">ceph orch ls <span class="o">[</span>--service_type type<span class="o">]</span> <span class="o">[</span>--service_name name<span class="o">]</span> <span class="o">[</span>--export<span class="o">]</span> <span class="o">[</span>--format f<span class="o">]</span> <span class="o">[</span>--refresh<span class="o">]</span></span>
</pre></div></div></div></blockquote>
<p>Discover the status of a particular service or daemon:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch ls --service_type <span class="nb">type</span> --service_name &lt;name&gt; <span class="o">[</span>--refresh<span class="o">]</span></span>
</pre></div></div></div></blockquote>
<p>To export the service specifications knows to the orchestrator, run the following command.</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch ls --export</span>
</pre></div></div></div></blockquote>
<p>The service specifications exported with this command will be exported as yaml
and that yaml can be used with the <code class="docutils literal notranslate"><span class="pre">ceph</span> <span class="pre">orch</span> <span class="pre">apply</span> <span class="pre">-i</span></code> command.</p>
<p>For information about retrieving the specifications of single services (including examples of commands), see <a class="reference internal" href="#orchestrator-cli-service-spec-retrieve"><span class="std std-ref">Retrieving the running Service Specification</span></a>.</p>
</div>
<div class="section" id="daemon-status">
<h2>Daemon Status<a class="headerlink" href="#daemon-status" title="Permalink to this headline">¶</a></h2>
<p>A daemon is a systemd unit that is running and part of a service.</p>
<p>To see the status of a daemon, do the following:</p>
<ol class="arabic simple">
<li><p>Print a list of all daemons known to the orchestrator.</p></li>
<li><p>Query the status of the target daemon.</p></li>
</ol>
<p>First, print a list of all daemons known to the orchestrator:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch ps <span class="o">[</span>--hostname host<span class="o">]</span> <span class="o">[</span>--daemon_type type<span class="o">]</span> <span class="o">[</span>--service_name name<span class="o">]</span> <span class="o">[</span>--daemon_id id<span class="o">]</span> <span class="o">[</span>--format f<span class="o">]</span> <span class="o">[</span>--refresh<span class="o">]</span></span>
</pre></div></div></div></blockquote>
<p>Then query the status of a particular service instance (mon, osd, mds, rgw).
For OSDs the id is the numeric OSD ID. For MDS services the id is the file
system name:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch ps --daemon_type osd --daemon_id <span class="m">0</span></span>
</pre></div></div></div></blockquote>
</div>
<div class="section" id="service-specification">
<span id="orchestrator-cli-service-spec"></span><h2>Service Specification<a class="headerlink" href="#service-specification" title="Permalink to this headline">¶</a></h2>
<p>A <em>Service Specification</em> is a data structure that is used to specify the
deployment of services.  Here is an example of a service specification in YAML:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">rgw</span>
<span class="nt">service_id</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">realm.zone</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">hosts</span><span class="p">:</span>
    <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host1</span>
    <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host2</span>
    <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host3</span>
<span class="nt">unmanaged</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">false</span>
<span class="nt">networks</span><span class="p">:</span>
<span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">192.169.142.0/24</span>
<span class="nt">spec</span><span class="p">:</span>
  <span class="c1"># Additional service specific attributes.</span>
</pre></div>
</div>
<p>In this example, the properties of this service specification are:</p>
<dl class="py class">
<dt id="ceph.deployment.service_spec.ServiceSpec">
<em class="property"><span class="pre">class</span> </em><code class="sig-prename descclassname"><span class="pre">ceph.deployment.service_spec.</span></code><code class="sig-name descname"><span class="pre">ServiceSpec</span></code><span class="sig-paren">(</span><em class="sig-param"><span class="n"><span class="pre">service_type</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">service_id</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">placement</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">count</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">config</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">unmanaged</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">preview_only</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">False</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">networks</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em>, <em class="sig-param"><span class="n"><span class="pre">extra_container_args</span></span><span class="o"><span class="pre">=</span></span><span class="default_value"><span class="pre">None</span></span></em><span class="sig-paren">)</span><a class="headerlink" href="#ceph.deployment.service_spec.ServiceSpec" title="Permalink to this definition">¶</a></dt>
<dd><p>Details of service creation.</p>
<p>Request to the orchestrator for a cluster of daemons
such as MDS, RGW, iscsi gateway, MONs, MGRs, Prometheus</p>
<p>This structure is supposed to be enough information to
start the services.</p>
<dl class="py attribute">
<dt id="ceph.deployment.service_spec.ServiceSpec.networks">
<code class="sig-name descname"><span class="pre">networks</span></code><em class="property"><span class="pre">:</span> <span class="pre">List</span><span class="p"><span class="pre">[</span></span><span class="pre">str</span><span class="p"><span class="pre">]</span></span></em><a class="headerlink" href="#ceph.deployment.service_spec.ServiceSpec.networks" title="Permalink to this definition">¶</a></dt>
<dd><p>A list of network identities instructing the daemons to only bind
on the particular networks in that list. In case the cluster is distributed
across multiple networks, you can add multiple networks. See
<a class="reference internal" href="monitoring/#cephadm-monitoring-networks-ports"><span class="std std-ref">Networks and Ports</span></a>,
<a class="reference internal" href="rgw/#cephadm-rgw-networks"><span class="std std-ref">Specifying Networks</span></a> and <a class="reference internal" href="mgr/#cephadm-mgr-networks"><span class="std std-ref">Specifying Networks</span></a>.</p>
</dd></dl>

<dl class="py attribute">
<dt id="ceph.deployment.service_spec.ServiceSpec.placement">
<code class="sig-name descname"><span class="pre">placement</span></code><em class="property"><span class="pre">:</span> <a class="reference internal" href="../../mgr/orchestrator_modules/#ceph.deployment.service_spec.PlacementSpec" title="ceph.deployment.service_spec.PlacementSpec"><span class="pre">ceph.deployment.service_spec.PlacementSpec</span></a></em><a class="headerlink" href="#ceph.deployment.service_spec.ServiceSpec.placement" title="Permalink to this definition">¶</a></dt>
<dd><p>See <a class="reference internal" href="#orchestrator-cli-placement-spec"><span class="std std-ref">Daemon Placement</span></a>.</p>
</dd></dl>

<dl class="py attribute">
<dt id="ceph.deployment.service_spec.ServiceSpec.service_id">
<code class="sig-name descname"><span class="pre">service_id</span></code><a class="headerlink" href="#ceph.deployment.service_spec.ServiceSpec.service_id" title="Permalink to this definition">¶</a></dt>
<dd><p>The name of the service. Required for <code class="docutils literal notranslate"><span class="pre">iscsi</span></code>, <code class="docutils literal notranslate"><span class="pre">mds</span></code>, <code class="docutils literal notranslate"><span class="pre">nfs</span></code>, <code class="docutils literal notranslate"><span class="pre">osd</span></code>, <code class="docutils literal notranslate"><span class="pre">rgw</span></code>,
<code class="docutils literal notranslate"><span class="pre">container</span></code>, <code class="docutils literal notranslate"><span class="pre">ingress</span></code></p>
</dd></dl>

<dl class="py attribute">
<dt id="ceph.deployment.service_spec.ServiceSpec.service_type">
<code class="sig-name descname"><span class="pre">service_type</span></code><a class="headerlink" href="#ceph.deployment.service_spec.ServiceSpec.service_type" title="Permalink to this definition">¶</a></dt>
<dd><p>The type of the service. Needs to be either a Ceph
service (<code class="docutils literal notranslate"><span class="pre">mon</span></code>, <code class="docutils literal notranslate"><span class="pre">crash</span></code>, <code class="docutils literal notranslate"><span class="pre">mds</span></code>, <code class="docutils literal notranslate"><span class="pre">mgr</span></code>, <code class="docutils literal notranslate"><span class="pre">osd</span></code> or
<code class="docutils literal notranslate"><span class="pre">rbd-mirror</span></code>), a gateway (<code class="docutils literal notranslate"><span class="pre">nfs</span></code> or <code class="docutils literal notranslate"><span class="pre">rgw</span></code>), part of the
monitoring stack (<code class="docutils literal notranslate"><span class="pre">alertmanager</span></code>, <code class="docutils literal notranslate"><span class="pre">grafana</span></code>, <code class="docutils literal notranslate"><span class="pre">node-exporter</span></code> or
<code class="docutils literal notranslate"><span class="pre">prometheus</span></code>) or (<code class="docutils literal notranslate"><span class="pre">container</span></code>) for custom containers.</p>
</dd></dl>

<dl class="py attribute">
<dt id="ceph.deployment.service_spec.ServiceSpec.unmanaged">
<code class="sig-name descname"><span class="pre">unmanaged</span></code><a class="headerlink" href="#ceph.deployment.service_spec.ServiceSpec.unmanaged" title="Permalink to this definition">¶</a></dt>
<dd><p>If set to <code class="docutils literal notranslate"><span class="pre">true</span></code>, the orchestrator will not deploy nor remove
any daemon associated with this service. Placement and all other properties
will be ignored. This is useful, if you do not want this service to be
managed temporarily. For cephadm, See <a class="reference internal" href="#cephadm-spec-unmanaged"><span class="std std-ref">Disabling automatic deployment of daemons</span></a></p>
</dd></dl>

</dd></dl>

<p>Each service type can have additional service-specific properties.</p>
<p>Service specifications of type <code class="docutils literal notranslate"><span class="pre">mon</span></code>, <code class="docutils literal notranslate"><span class="pre">mgr</span></code>, and the monitoring
types do not require a <code class="docutils literal notranslate"><span class="pre">service_id</span></code>.</p>
<p>A service of type <code class="docutils literal notranslate"><span class="pre">osd</span></code> is described in <a class="reference internal" href="osd/#drivegroups"><span class="std std-ref">Advanced OSD Service Specifications</span></a></p>
<p>Many service specifications can be applied at once using <code class="docutils literal notranslate"><span class="pre">ceph</span> <span class="pre">orch</span> <span class="pre">apply</span> <span class="pre">-i</span></code>
by submitting a multi-document YAML file:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="n">cat</span> <span class="o">&lt;&lt;</span><span class="n">EOF</span> <span class="o">|</span> <span class="n">ceph</span> <span class="n">orch</span> <span class="n">apply</span> <span class="o">-</span><span class="n">i</span> <span class="o">-</span>
<span class="n">service_type</span><span class="p">:</span> <span class="n">mon</span>
<span class="n">placement</span><span class="p">:</span>
  <span class="n">host_pattern</span><span class="p">:</span> <span class="s2">&quot;mon*&quot;</span>
<span class="o">---</span>
<span class="n">service_type</span><span class="p">:</span> <span class="n">mgr</span>
<span class="n">placement</span><span class="p">:</span>
  <span class="n">host_pattern</span><span class="p">:</span> <span class="s2">&quot;mgr*&quot;</span>
<span class="o">---</span>
<span class="n">service_type</span><span class="p">:</span> <span class="n">osd</span>
<span class="n">service_id</span><span class="p">:</span> <span class="n">default_drive_group</span>
<span class="n">placement</span><span class="p">:</span>
  <span class="n">host_pattern</span><span class="p">:</span> <span class="s2">&quot;osd*&quot;</span>
<span class="n">data_devices</span><span class="p">:</span>
  <span class="nb">all</span><span class="p">:</span> <span class="n">true</span>
<span class="n">EOF</span>
</pre></div>
</div>
<div class="section" id="retrieving-the-running-service-specification">
<span id="orchestrator-cli-service-spec-retrieve"></span><h3>Retrieving the running Service Specification<a class="headerlink" href="#retrieving-the-running-service-specification" title="Permalink to this headline">¶</a></h3>
<p>If the services have been started via <code class="docutils literal notranslate"><span class="pre">ceph</span> <span class="pre">orch</span> <span class="pre">apply...</span></code>, then directly changing
the Services Specification is complicated. Instead of attempting to directly change
the Services Specification, we suggest exporting the running Service Specification by
following these instructions:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch ls --service-name rgw.&lt;realm&gt;.&lt;zone&gt; --export &gt; rgw.&lt;realm&gt;.&lt;zone&gt;.yaml</span>
<span class="prompt1">ceph orch ls --service-type mgr --export &gt; mgr.yaml</span>
<span class="prompt1">ceph orch ls --export &gt; cluster.yaml</span>
</pre></div></div></div></blockquote>
<p>The Specification can then be changed and re-applied as above.</p>
</div>
<div class="section" id="updating-service-specifications">
<h3>Updating Service Specifications<a class="headerlink" href="#updating-service-specifications" title="Permalink to this headline">¶</a></h3>
<p>The Ceph Orchestrator maintains a declarative state of each
service in a <code class="docutils literal notranslate"><span class="pre">ServiceSpec</span></code>. For certain operations, like updating
the RGW HTTP port, we need to update the existing
specification.</p>
<ol class="arabic">
<li><p>List the current <code class="docutils literal notranslate"><span class="pre">ServiceSpec</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch ls --service_name<span class="o">=</span>&lt;service-name&gt; --export &gt; myservice.yaml</span>
</pre></div></div></li>
<li><p>Update the yaml file:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">vi myservice.yaml</span>
</pre></div></div></li>
<li><p>Apply the new <code class="docutils literal notranslate"><span class="pre">ServiceSpec</span></code>:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch apply -i myservice.yaml <span class="o">[</span>--dry-run<span class="o">]</span></span>
</pre></div></div></li>
</ol>
</div>
</div>
<div class="section" id="daemon-placement">
<span id="orchestrator-cli-placement-spec"></span><h2>Daemon Placement<a class="headerlink" href="#daemon-placement" title="Permalink to this headline">¶</a></h2>
<p>For the orchestrator to deploy a <em>service</em>, it needs to know where to deploy
<em>daemons</em>, and how many to deploy.  This is the role of a placement
specification.  Placement specifications can either be passed as command line arguments
or in a YAML files.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>cephadm will not deploy daemons on hosts with the <code class="docutils literal notranslate"><span class="pre">_no_schedule</span></code> label; see <a class="reference internal" href="../host-management/#cephadm-special-host-labels"><span class="std std-ref">Special host labels</span></a>.</p>
</div>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>The <strong>apply</strong> command can be confusing. For this reason, we recommend using
YAML specifications.</p>
<p>Each <code class="docutils literal notranslate"><span class="pre">ceph</span> <span class="pre">orch</span> <span class="pre">apply</span> <span class="pre">&lt;service-name&gt;</span></code> command supersedes the one before it.
If you do not use the proper syntax, you will clobber your work
as you go.</p>
<p>For example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch apply mon host1</span>
<span class="prompt1">ceph orch apply mon host2</span>
<span class="prompt1">ceph orch apply mon host3</span>
</pre></div></div><p>This results in only one host having a monitor applied to it: host 3.</p>
<p>(The first command creates a monitor on host1. Then the second command
clobbers the monitor on host1 and creates a monitor on host2. Then the
third command clobbers the monitor on host2 and creates a monitor on
host3. In this scenario, at this point, there is a monitor ONLY on
host3.)</p>
<p>To make certain that a monitor is applied to each of these three hosts,
run a command like this:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch apply mon <span class="s2">&quot;host1,host2,host3&quot;</span></span>
</pre></div></div><p>There is another way to apply monitors to multiple hosts: a <code class="docutils literal notranslate"><span class="pre">yaml</span></code> file
can be used. Instead of using the “ceph orch apply mon” commands, run a
command of this form:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch apply -i file.yaml</span>
</pre></div></div><p>Here is a sample <strong>file.yaml</strong> file</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">mon</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">hosts</span><span class="p">:</span>
   <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host1</span>
   <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host2</span>
   <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host3</span>
</pre></div>
</div>
</div>
<div class="section" id="explicit-placements">
<h3>Explicit placements<a class="headerlink" href="#explicit-placements" title="Permalink to this headline">¶</a></h3>
<p>Daemons can be explicitly placed on hosts by simply specifying them:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">orch apply prometheus --placement<span class="o">=</span><span class="s2">&quot;host1 host2 host3&quot;</span></span>
</pre></div></div></div></blockquote>
<p>Or in YAML:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">prometheus</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">hosts</span><span class="p">:</span>
    <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host1</span>
    <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host2</span>
    <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host3</span>
</pre></div>
</div>
<p>MONs and other services may require some enhanced network specifications:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">orch daemon add mon --placement<span class="o">=</span><span class="s2">&quot;myhost:[v2:1.2.3.4:3300,v1:1.2.3.4:6789]=name&quot;</span></span>
</pre></div></div></div></blockquote>
<p>where <code class="docutils literal notranslate"><span class="pre">[v2:1.2.3.4:3300,v1:1.2.3.4:6789]</span></code> is the network address of the monitor
and <code class="docutils literal notranslate"><span class="pre">=name</span></code> specifies the name of the new monitor.</p>
</div>
<div class="section" id="placement-by-labels">
<span id="orch-placement-by-labels"></span><h3>Placement by labels<a class="headerlink" href="#placement-by-labels" title="Permalink to this headline">¶</a></h3>
<p>Daemon placement can be limited to hosts that match a specific label. To set
a label <code class="docutils literal notranslate"><span class="pre">mylabel</span></code> to the appropriate hosts, run this command:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch host label add *&lt;hostname&gt;* mylabel</span>
</pre></div></div><p>To view the current hosts and labels, run this command:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch host ls</span>
</pre></div></div><p>For example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch host label add host1 mylabel</span>
<span class="prompt1">ceph orch host label add host2 mylabel</span>
<span class="prompt1">ceph orch host label add host3 mylabel</span>
<span class="prompt1">ceph orch host ls</span>
</pre></div></div><div class="highlight-bash notranslate"><div class="highlight"><pre><span></span>HOST   ADDR   LABELS  STATUS
host1         mylabel
host2         mylabel
host3         mylabel
host4
host5
</pre></div>
</div>
</div></blockquote>
<p>Now, Tell cephadm to deploy daemons based on the label by running
this command:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">orch apply prometheus --placement<span class="o">=</span><span class="s2">&quot;label:mylabel&quot;</span></span>
</pre></div></div></div></blockquote>
<p>Or in YAML:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">prometheus</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">label</span><span class="p">:</span> <span class="s">&quot;mylabel&quot;</span>
</pre></div>
</div>
<ul class="simple">
<li><p>See <a class="reference internal" href="../host-management/#orchestrator-host-labels"><span class="std std-ref">Host labels</span></a></p></li>
</ul>
</div>
<div class="section" id="placement-by-pattern-matching">
<h3>Placement by pattern matching<a class="headerlink" href="#placement-by-pattern-matching" title="Permalink to this headline">¶</a></h3>
<p>Daemons can be placed on hosts as well:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">orch apply prometheus --placement<span class="o">=</span><span class="s1">&#39;myhost[1-3]&#39;</span></span>
</pre></div></div></div></blockquote>
<p>Or in YAML:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">prometheus</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">host_pattern</span><span class="p">:</span> <span class="s">&quot;myhost[1-3]&quot;</span>
</pre></div>
</div>
<p>To place a service on <em>all</em> hosts, use <code class="docutils literal notranslate"><span class="pre">&quot;*&quot;</span></code>:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">orch apply node-exporter --placement<span class="o">=</span><span class="s1">&#39;*&#39;</span></span>
</pre></div></div></div></blockquote>
<p>Or in YAML:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">node-exporter</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">host_pattern</span><span class="p">:</span> <span class="s">&quot;*&quot;</span>
</pre></div>
</div>
</div>
<div class="section" id="changing-the-number-of-daemons">
<h3>Changing the number of daemons<a class="headerlink" href="#changing-the-number-of-daemons" title="Permalink to this headline">¶</a></h3>
<p>By specifying <code class="docutils literal notranslate"><span class="pre">count</span></code>, only the number of daemons specified will be created:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">orch apply prometheus --placement<span class="o">=</span><span class="m">3</span></span>
</pre></div></div></div></blockquote>
<p>To deploy <em>daemons</em> on a subset of hosts, specify the count:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">orch apply prometheus --placement<span class="o">=</span><span class="s2">&quot;2 host1 host2 host3&quot;</span></span>
</pre></div></div></div></blockquote>
<p>If the count is bigger than the amount of hosts, cephadm deploys one per host:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">orch apply prometheus --placement<span class="o">=</span><span class="s2">&quot;3 host1 host2&quot;</span></span>
</pre></div></div></div></blockquote>
<p>The command immediately above results in two Prometheus daemons.</p>
<p>YAML can also be used to specify limits, in the following way:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">prometheus</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">count</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">3</span>
</pre></div>
</div>
<p>YAML can also be used to specify limits on hosts:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">prometheus</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">count</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">2</span>
  <span class="nt">hosts</span><span class="p">:</span>
    <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host1</span>
    <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host2</span>
    <span class="p p-Indicator">-</span> <span class="l l-Scalar l-Scalar-Plain">host3</span>
</pre></div>
</div>
</div>
<div class="section" id="algorithm-description">
<h3>Algorithm description<a class="headerlink" href="#algorithm-description" title="Permalink to this headline">¶</a></h3>
<p>Cephadm’s declarative state consists of a list of service specifications
containing placement specifications.</p>
<p>Cephadm continually compares a list of daemons actually running in the cluster
against the list in the service specifications. Cephadm adds new daemons and
removes old daemons as necessary in order to conform to the service
specifications.</p>
<p>Cephadm does the following to maintain compliance with the service
specifications.</p>
<p>Cephadm first selects a list of candidate hosts. Cephadm seeks explicit host
names and selects them. If cephadm finds no explicit host names, it looks for
label specifications. If no label is defined in the specification, cephadm
selects hosts based on a host pattern. If no host pattern is defined, as a last
resort, cephadm selects all known hosts as candidates.</p>
<p>Cephadm is aware of existing daemons running services and tries to avoid moving
them.</p>
<p>Cephadm supports the deployment of a specific amount of services.
Consider the following service specification:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">mds</span>
<span class="nt">service_name</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">myfs</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">count</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">3</span>
  <span class="nt">label</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">myfs</span>
</pre></div>
</div>
<p>This service specification instructs cephadm to deploy three daemons on hosts
labeled <code class="docutils literal notranslate"><span class="pre">myfs</span></code> across the cluster.</p>
<p>If there are fewer than three daemons deployed on the candidate hosts, cephadm
randomly chooses hosts on which to deploy new daemons.</p>
<p>If there are more than three daemons deployed on the candidate hosts, cephadm
removes existing daemons.</p>
<p>Finally, cephadm removes daemons on hosts that are outside of the list of
candidate hosts.</p>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>There is a special case that cephadm must consider.</p>
<p>If there are fewer hosts selected by the placement specification than
demanded by <code class="docutils literal notranslate"><span class="pre">count</span></code>, cephadm will deploy only on the selected hosts.</p>
</div>
</div>
</div>
<div class="section" id="removing-a-service">
<span id="orch-rm"></span><h2>Removing a Service<a class="headerlink" href="#removing-a-service" title="Permalink to this headline">¶</a></h2>
<p>In order to remove a service including the removal
of all daemons of that service, run</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><style type="text/css">
span.prompt2:before {
  content: "$ ";
}
</style><span class="prompt2">ceph orch rm &lt;service-name&gt;</span>
</pre></div></div><p>For example:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt2">ceph orch rm rgw.myrgw</span>
</pre></div></div></div>
<div class="section" id="disabling-automatic-deployment-of-daemons">
<span id="cephadm-spec-unmanaged"></span><h2>Disabling automatic deployment of daemons<a class="headerlink" href="#disabling-automatic-deployment-of-daemons" title="Permalink to this headline">¶</a></h2>
<p>Cephadm supports disabling the automated deployment and removal of daemons on a
per service basis. The CLI supports two commands for this.</p>
<p>In order to fully remove a service, see <a class="reference internal" href="#orch-rm"><span class="std std-ref">Removing a Service</span></a>.</p>
<div class="section" id="disabling-automatic-management-of-daemons">
<h3>Disabling automatic management of daemons<a class="headerlink" href="#disabling-automatic-management-of-daemons" title="Permalink to this headline">¶</a></h3>
<p>To disable the automatic management of dameons, set <code class="docutils literal notranslate"><span class="pre">unmanaged=True</span></code> in the
<a class="reference internal" href="#orchestrator-cli-service-spec"><span class="std std-ref">Service Specification</span></a> (<code class="docutils literal notranslate"><span class="pre">mgr.yaml</span></code>).</p>
<p><code class="docutils literal notranslate"><span class="pre">mgr.yaml</span></code>:</p>
<div class="highlight-yaml notranslate"><div class="highlight"><pre><span></span><span class="nt">service_type</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">mgr</span>
<span class="nt">unmanaged</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">true</span>
<span class="nt">placement</span><span class="p">:</span>
  <span class="nt">label</span><span class="p">:</span> <span class="l l-Scalar l-Scalar-Plain">mgr</span>
</pre></div>
</div>
<div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch apply -i mgr.yaml</span>
</pre></div></div><div class="admonition note">
<p class="admonition-title">Note</p>
<p>After you apply this change in the Service Specification, cephadm will no
longer deploy any new daemons (even if the placement specification matches
additional hosts).</p>
</div>
</div>
<div class="section" id="deploying-a-daemon-on-a-host-manually">
<h3>Deploying a daemon on a host manually<a class="headerlink" href="#deploying-a-daemon-on-a-host-manually" title="Permalink to this headline">¶</a></h3>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>This workflow has a very limited use case and should only be used
in rare circumstances.</p>
</div>
<p>To manually deploy a daemon on a host, follow these steps:</p>
<p>Modify the service spec for a service by getting the
existing spec, adding <code class="docutils literal notranslate"><span class="pre">unmanaged:</span> <span class="pre">true</span></code>, and applying the modified spec.</p>
<p>Then manually deploy the daemon using the following:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch daemon add &lt;daemon-type&gt;  --placement<span class="o">=</span>&lt;placement spec&gt;</span>
</pre></div></div></div></blockquote>
<p>For example :</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch daemon add mgr --placement<span class="o">=</span>my_host</span>
</pre></div></div></div></blockquote>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>Removing <code class="docutils literal notranslate"><span class="pre">unmanaged:</span> <span class="pre">true</span></code> from the service spec will
enable the reconciliation loop for this service and will
potentially lead to the removal of the daemon, depending
on the placement spec.</p>
</div>
</div>
<div class="section" id="removing-a-daemon-from-a-host-manually">
<h3>Removing a daemon from a host manually<a class="headerlink" href="#removing-a-daemon-from-a-host-manually" title="Permalink to this headline">¶</a></h3>
<p>To manually remove a daemon, run a command of the following form:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch daemon rm &lt;daemon name&gt;... <span class="o">[</span>--force<span class="o">]</span></span>
</pre></div></div></div></blockquote>
<p>For example:</p>
<blockquote>
<div><div class="highlight-default notranslate"><div class="highlight"><pre><span class="prompt1">ceph orch daemon rm mgr.my_host.xyzxyz</span>
</pre></div></div></div></blockquote>
<div class="admonition note">
<p class="admonition-title">Note</p>
<p>For managed services (<code class="docutils literal notranslate"><span class="pre">unmanaged=False</span></code>), cephadm will automatically
deploy a new daemon a few seconds later.</p>
</div>
</div>
<div class="section" id="see-also">
<h3>See also<a class="headerlink" href="#see-also" title="Permalink to this headline">¶</a></h3>
<ul class="simple">
<li><p>See <a class="reference internal" href="osd/#cephadm-osd-declarative"><span class="std std-ref">Declarative State</span></a> for special handling of unmanaged OSDs.</p></li>
<li><p>See also <a class="reference internal" href="../troubleshooting/#cephadm-pause"><span class="std std-ref">Pausing or disabling cephadm</span></a></p></li>
</ul>
</div>
</div>
</div>



           </div>
           
          </div>
          <footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="footer navigation">
        <a href="mon/" class="btn btn-neutral float-right" title="MON Service" accesskey="n" rel="next">Next <span class="fa fa-arrow-circle-right" aria-hidden="true"></span></a>
        <a href="../host-management/" class="btn btn-neutral float-left" title="Host Management" accesskey="p" rel="prev"><span class="fa fa-arrow-circle-left" aria-hidden="true"></span> Previous</a>
    </div>

  <hr/>

  <div role="contentinfo">
    <p>
        &#169; Copyright 2016, Ceph authors and contributors. Licensed under Creative Commons Attribution Share Alike 3.0 (CC-BY-SA-3.0).

    </p>
  </div> 

</footer>
        </div>
      </div>

    </section>

  </div>
  

  <script type="text/javascript">
      jQuery(function () {
          SphinxRtdTheme.Navigation.enable(true);
      });
  </script>

  
  
    
   

</body>
</html>